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ABSTRACT 


This study evaluates the accuracy of an established 
reliability measurement procedure (NAVWEPS OD 29304) by 
computer simulation. The reliability measurement procedure 
assumes components fail according to an Exponential Failure 
Law. This study tests the accuracy of that procedure when 
components obey a Weibull Failure Law or a Log Normal Fail- 


ure Law, 
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COS Perak 1 
INTRODUCTION 


The highly complex weapon systems being developed 
for the Navy today require a rapid assessment of system and 
subsystem reliability during research and development, and 
production phases. Currently there exists a statistical 
model, the Guide Manual For Reliability Measurement Program 
(NAVWEPS OD 29304), that "...can be utilized by all contract- 
ors for subsystem reliability measurement and by the Navy 
for weapon system reliability measurement, "1 This model is 
versatile in that it permits the combination of test data 
from all levels and can be continuously updated as new test 
data becomes available. The model is thus a rapid approx- 
imation procedure for determining system reliability. How- 
ever, the procedure has some restrictive assumptions which 
must be kept in mind, 

One notable restrictive assumption is that all compo- 
nents are assumed to have a constant failure rate. This may 
not be necessarily true, although many times is a good 
approximation. This leads to the commonly used Exponential 
Failure Law. However, suppose the failures were actually 
given by a Weibull or a Log Normal Failure Law. Then this 


procedure must adequately estimate the system or subsystem 


louide Manual For Reliability Measurement Program 


(NAVWEPS OD 29304, 15 May 1965), p. 3-1. 


reliability under the Exponential Law assumption. 

The purpose of this study is to evaluate the accuracy 
of the procedure in NAVWEPS OD 29304 when failures actually 
occur in a weibull and Log Normal fashion. 

The approximation procedure uses statistical estimates 
of failure rates based on sample data and are thus subject 
to statistical uncertainty. Therefore, at best, the pro- ¢ 
cedure must yield a lower confidence limit on system or 
subsystem reliability, or an upper limit on failure rate. 
It is desired to test the accuracy of this procedure by 
Simulating the distribution of the lower confidence limit 
for various systems and comparing the simulation results 
with the true reliability. 

The method of simulation involves obtaining random 
Weibull and Log Normal failure time variates for components 
in a given system with a true system reliability, Re- 

These quantities represent the times to failure of compo- 
nents provided they are less than the duration of the test 
(i.e. the planned test time). MThen, using the statistical 


Program, a 100(1l-a)% lower confidence limit, for 


R 
s,L(a)? 
Re is Obtained. A large number of Feplications of this 
pROCCSs are completed to construct 4 distri oucton sos i L(a)° 
9 
If the procedure is indeed accurate, the (1-a)t4 percentile 
point of the constructed distribution will be Roe That is, 


A 
ing Rs ,n(a) iS a true 100(1-a)% lower confidence limit for 


R 


“N 
ep lleiae 10S o 


me = l-a. If the (l -a)th percentile 
point of the distribution of R, 1(,) is denoted by A,_o> 
then A); _, should equal R,, regardless of the set of paran- 
eter values chosen. Thus a measure of the accuracy of the 
procedure is given by the quantity Ay ee = Re . Rio other 
measures of the accuracy of this procedure are the mean and 
standard deviation of the distribution of fee 

The accuracy of this procedure is examined fora 
variety of combinations of components and sets of parameter 
values. In all cases the systems under consideration con- 
sist of four components in series. All components are 
assumed to behave independently so the system reliability 
is the product of the four component reltabilities, 

AS expected, the closer the Weibull and hog Normal 
distributions approximate the Exponential, the better the 
accuracy of the procedure. However, for some components 
with failure time distributions differing considerably from 
the Exponential, the accuracy is fairly good using this 
method of reliability measurement. If the absolute differ- 
ence A, .q 7 Bg| is less than .0¢, the accuracy is called 
good. The accuracy of the procedure is good for planned 
test times of short duration and diminishes as they are 
increased, The study concludes that the reliability measure- 
ment procedure, being an approximation procedure, is useful 
for some systems following Weibull and Log Normal Failure 
Laws provided they do not deviate from the Exponential Fail- 


ure Law by too significant an amount. 


CHAPTER. 1 I 


Slat io LIC Aenea BLE TY MODEL“ 


The statistical Reliability Model provides the guide 
lines for reliability measurement on Navy weapons systems. 
The "Guide Manual" can be used by both contractors for sub- 
system reliability measurement and by the Navy for weapon 
system reliability measurement. A brief description of this 
model's assumptions and methodology are presented in this 
chapteaw-, 

The system reliability obtained is based upon testing 
components for a time duration, called the planned test time, 
or until failure occurs under a given stress condition. 
Also, the model provides for testing a system for component 
failure by the same method, 

This model considers components in a complex system 
which must operate successfully over a defined "mission", 
lasting a specified duration of time. Time can thus be 
measured in "mission units". System reliability is thus a 
function of environmental and usage stresses which include 
vibration, shock, etc., and the operating vs. non-operating 
conditions as well as a function of the associated part 
failure rate parameters together with the time duration of 


the environments and usage stresses. The reliability 


fguide Manual For Reliability Measurement Program 


(NAVWEPS OD 29304, 15 May 1965), pp. 3-1 to 3-15. 
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measurement system is dependent upon the following assump- 


tions: 


le. Constant failure rate, Theé@em@ponential failure 
law is assumed to hold, mainly for its common 
usage, mathematical simplicity, practical 
Simplicity, and reasonability. 


2. Additivity of stress effects. "The failure nate 
induced by two simultaneously acting stresses 
is equal to the sum of the failure rates due to 
the two stresses acting sequentially."3 Thus, 
mission experience can be simulated by adding 
data from separate environmental tests, 

3. Independence of component Talllumes, ”*...This.is 
assumed because components are normally tested 
individually by type, and subsystem reliability 
is estimated using component and other appli- 
cable test results." 

4, Failure rate constancy. "The failure rate is con- 
sidered a function of only the stress acting." 
In other words, the component being tested has 
no "memory" as to previous stresses of a differ- 
ent type. 

Computation of the system reliability is accomplished 

by Ghe following procedure: 

An unbiased failure rate estimate is computed, pro- 
viding the failure rates are anticipated as being small. 
For ease of computation, a simplified unbiased estimator, 
which is slightly conservative, is used. For the i% com- 


ras 
ponent, the unbiased estimator, hj is given by; 


apd. ple3-3. 
4opid. 


5Ibid. 


1 


~ See ee (1) 


- component (i.e. the 


where Ny = the sample size of the i” 
number of components of type i tested). 


T, = the test time (time to failure) of the component 


of type i. 

To4 = the planned test time for component i, 
and f = the number of N, failure times which are less 
than TOL? 


The system failure rate estimate is then given by: 
m N 
= > dk (2) 


where m is the number of components in the system. The 
component number assumes values i = 1,...,M. 


The variance of the unbiased failure rate estimate is 


given by; 
mi IN 
Variance = ¢£ (r./S. ) 
a 3 es 
shee di 
where 
Ny 
5S = 3g T = the sum of all test times 
1° 42) 43 


accumulated on the N components of type i. 


1 


These estimates of failure rates for components ina 
given test condition can be continually updated during the 
development phase as more data becomes available. Also, 
during any higher level of assemblage, further testing can 
also yield data to modify the failure rate estimate. Data 
is obtained indicating the time that each component operates 
during the system test. 


The reliability estimate for the system is then given 


NN 


™ = 
by Rg = e \ where x is the appropriate,failure rate esti- 


mate and only series systems or subsystems are considered, 
given by formula (2). 

The reliability estimates obtained are subject to 
svatistical uncertainty, therefore the important item to 
observe is the confidence interval about the estimate. 
meme, the lowem confidence limit. on system meld ability 
becomes the pertinent quantity to observe. The "Guide 
Manual" bases the lower confidence limit on Normal theory 
and it is corrected to compensate for small values of }. 

The upper limit on failure rate, - provides the oa 
ponding lower confidence limit on reliability by Ry pze Ue 


The upper limit on failure rate is given by: 


A ON ‘yi aaa LA2 m 
Chi + KC + 4HiK oc + Kec for x: f,>0 
2 i=. + 
A ] 
y= (3) 
Ke m m 
— ely >) mOr 2 he = 0 
nN i=1 1 i=l + 
Ne 
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wnere 


n = the number of component-environment-test condition 
terms in the summation, and K = the percentage point of the 
Normal Distribution. Tne values of K for a given confidence 
level are not completely appropriate for small values of }. 
Consequently, a correction factor, 8, is used to obtain 
desired precision. Beta values are tabulated for the 802 
confidence limit in reference (2). Thus the percentage point 
of the Normal Distribution, K, is modified by BK and replaced 
in formula (3). 

A more detailed description of this reliability meas- 
urement method plus examples of the procedure are given in 


reference (2). 


14 


CHAPTERe Iii 
SIMULATION PROCEDURE 


Suppose the assumption of constant failure rate used 
in the previously described statistical model is relaxed. 
It is now desired to test the accuracy of that model when 
failures occur according to a Weibull Failure Law and a Log 
Normal Failure Law. For testing purposes by simulation 
techniques, consider a series system with four components, 
each component having a well defined failure law. 

The true reliability for each component of this series 
system is given by R,, i = 1,2,3,4, and thus the true system 


al 
reliability is 


4 


Using the statistical model, an estimate of the (1-a)™ 


percentile confidence limit of the systen, can be 


2 
s,L(a)? 
obtained by computer simulation. This is a random variable 


as calculated from the statistical model. [If 
P( R > R ) aL 
ee ne) = 


aN 
then, in fact, Re L(a) is an exact (wey percentile lower 
3 
confidence limit for R,- This says that R, is always the 
(l1-a)*" percentile point of the probability distribution of 


A A 
Rs sL(a)° The distribution of Bs L(a) is constructed by 


5 


computer simulation. Letting A be the ae oe per- 


l=-a 
“N 
centile point of the distribution of Bee Lia)? then the 
9 
absolute difference, ee) “= R.| » 1S a measure of the 


accuracy of the statistical model. 

Two other measures of the accuracy of this model are 
the estimated mean, Bo 1(q)) and the estimated standard 
deviation, tor Which give some assurance that the 
actual values generated by the procedure are reasonable. 

The simulation procedure itself is now discussed for 
the series system of four components: 

The distribution of R, ,(,) 1s constructed by gener- 
ating 500 random observations on BS n(a) for a given systen, 


number of components tested, and reliability of the com- 


ponents. For this study a four component series system is 


used, with each component having reliability R,, i = 1,253 


ime-reliabilicy, Ry > is defined for each component by 


R, = R, (1) = P( T, >1), where T, is the time to failure 


i i 

random variable for a component of type i. This establishes 
the parameter values for a particular failure rate distri- 
bution. For example, if a component is to obey a Weibull 


Failure Law, 


R(t) = en (At)F (5) 
8 
and R(1l) =e = .995, then, for 8 = 2, this implies that 
i = . O07. 


Similarly, for the Log Normal Failure Law, if 


16 


a(t) = 2-2 - G [antl (6) 


C 


and R(1) = 1 - Q(-*/e ) = .995, then, for «= 1, this 
implies that A= 2.576. The failure rate functions deter- 
mined from the reliabilities and selected Wenuorene are 
Gasplayed in Appendix I. 

A random number generator is used to obtain a sample 
of Ny tested components of type i wnich fail according to 
their respective failure rate distributions. This Monte 
Carlo method draws a random number wnich is Uniformly Dis- 
tributed (0,1) and converts it ee a Weibull variate ora 
Log Normal variate to obtain tne time to failure of the ae 


a B 
component, ie For the Welbull Distrvbution, «< (AT) = Y, 


where Y is distributed as Uniform (0,1), implies that 


(-1n yyi/e 
X 


T = (7) 


is a random Weibull variate. For the Log Normal Distri- 
bution, if X is a random variable distributed as Normal(0,1l), 
then Z = CX + is a random variable distributed as Normal 
oo") and thus T = e4 is a random Log Normal variate. In 
this manner, Ny failure times are generated for each of the 
four components: 

Th Ty2 T) 3 ye a 

i722 a aalaaieg 

T 37 T 3 T33 ee a! "INN, 

Tar Tye Ty3 + * + Tony 


Using these failure times in the statistical procedure 
discussed in the previous chapter, values of a and B 
iS s,L(a) 
are obtained with that model, Each Ty j is examined for 
failure before the end of the planned test time or else 
terminated at the end of the planned test time. This same 
procedure is completed 500 times to generate 500 random 
A 
observations, thus constructing the distributions of R, 
a A 


The mean and standard deviation of the distributions 


are determined by: 


~~ 500 

A és 1 a N 

Re,L(a) ~ 500 .ty ss Ela), (8) 
es 500 

ie | 8 (9) 


(10) 


(lly) 





The 500 values of 8, 1 (q) are then ordered and the 
(l-a)™ percentile point of the distribution is obtained. 


Thus, the value A is available for comparison with Ro 


l1-a 
This process is done merely by counting down to the 10 


A 
value of AS Lia) 


80° percentile point. 


1st 


from the largest, which represents the 
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Several cases are examined with various failure rate 
distributions and with various combinations of components. 
For each case examined, the planned test time, Toa? varies 
to produce an optimistic» or pessimistic reliability test 
dle ~5, the system will yield an optimistic test since 
the planned test time will be reached before most components 
fail. The opposite will occur when Pot eee A TOA chosen 
such that the average failure rate is equal to the failure 
rate at time one appears to be a good test of system relia- 


bility for this procedure. That is, make i such that it 


satisfies the equation 


Toi 
== Z,(t)dt = Zz, (1) (12) 


where z(t) is the failure rate function; the failure distri- 
Bution divided by the reliability function: z(t)=f(t)/R(t). 

All cases studied by simulation procedures are given in 
Table I. All simulation results of these cases are given in 


fave Tl 


MN 


CAS 


MM. 
been 


CASES STUDIED BY SIMULATION PROCEDURES 


Compo- Failure rameters 
| se | ET | meres _{ A 


a (1 Sa AR 


TABLE I 


Weibull 
Weibull 
Log Normal 


Log Normal 


Neibull 
Neibull 
Log Normal 


Log Normal 


Weibull 
Weibull 
Log Normal 


Log Normal 


Weibull 
Weibull 
Log Normal 


Log Normal 


Weibull 
Weibull 
Weibull 


Weibull 


=a A=. 


0293 
0707 


ia A=. 


= Or pyc /S 


al ee A= 


0, M=5.152 


0293 


-1.5, a= sees 
0,M=5.15¢e 
= 20, M=5- 


1 
Pee J. 


= 055 M= 


152 


0378 
0378 
30345 


= 15S » M=3-6 345 


oplle 
p=1. 


C=1 


G=l. 


2, \=.0485 


2, h=.0485 


055 AbM=2.919 


59 KM=e2.919 


ee OL 5 
=.0485 
5 A=, O65 
=.0485 





Case Compo- 
No. nent No. 
6 


















TABLE I (Continued) 


Failure IB: 
| Fature | paraneters | | 


Tae p=e.919 7 


Gi led bM=e.919 
867 


Log Normal 





Log Normal 








OC es Nomemeel. iil Cir eceSyeee th e599 


to 1 as M=2.919 








Log Normal 





Weibull 
Weibull 
Weibull 


Neibull 





Log Normal | €=2.0, »p=4.0 912 








Log Normal | ©=2.0, w=4.0 


Log Normal 





Ge 200 





Log Normal Cua Zee M=H.0 


fal 


Coane ian Lv 
RESULTS AND CONCLUSIONS 


In the preceding chapters the statistical procedure 
from the Guide Manual For Reliability Measurement Program 
has been briefly explained to obtain a 100(1-a)% lower con- 
fidence limit on system reliability. The simulation of this 
method on a computer has been explained and the technique 
for measuring the accuracy of the statistical procedure 
stated. Now, from the results of the simulation, some con- 
clusions may be drawn. 

The accuracy of the statistical procedure has been 
examined for eight cases which represent different combi- 
nations of components for various sets of parameter values, 
The results for these cases are given in Table II. The 
mean and standard deviation of the 500 values of R. 1 (q) 
and Bo are given, along with the (l-a)th percentile point, 
Aj -qg- Im all cases studied, a was taken to be .20. 

The accuracy of the procedure is measured by the 


absolute difference between the gq th 


percentile of the ditee 
tribution of Be 1.20)? which is A 80? and the actual system 
reliability, R, (i.e. accuracy = A 89 = R. )e 

Also stated in the results, is the quantity TT, which 
is the amount of testing relative to the component un- 
reliabilities. In other words, this is a measure of the 
amount of testing required to achieve a desired accuracy 


for a given system with component failure rates, Z,(1), 


Ze 





and planned test times, T Them TT is given br 


oi° 


Z Ny2 (1) oy (13) 


As an example of the accuracy study, consider Case l, 
Where the system consists of four components in series dis- 


playing the following failure laws; 


Component J- Weibull, s=1.5, rA=.0295 
Ceompepen> 2:2 wWedibull, g= 2.05 &m=@0707 
Component 3: Log Normal, G=1.0, m=2.576 
Component 4: Log Normal, o=2.0, w= 5.152 


The component reliabilities, R, = .995, for all four 
components, so the system reliability, R. = -9801. The 
measure of accuracy for the optimistic planned test time, 
4 = a, is A 89 — ie = .008 for 500 components of each 
type tested. The pessimistic planned test time, Toi = pre 
yields an accuracy of .055 for 100 components of each type 


tested. The ad hoc planned test times, T eee 5, To2 = Zee, 


ol 
793 = 1.95, and Toy = 3.25, as determined from formula (12), 
give an accuracy of .030 for Ny = 100. “Pte accuracy for 
the ad hoc planned test times is only fair. This planned 
test time represents the best measure of accuracy of the 
three selected. 

It can be seen from Table II and Appendix I that the 
accuracy of the procedure is a function of how closely the 


Weibull and Log Normal failure rate functions approximate 


a constant failure rate function. The greater the Weibull 


23 


and Log Normal failure rate functions deviate from the 
Sencvany tarlure rave, the more inaccurate the procedure. 
Also, the accuracy decreases as the planned test time is 
increasea, —liewwetbulisfailure 2,abe ianction increases in 
time and the Log Normal failure rate function increases 
then decreases in time, thus deviating from the constant 
failure rate for large planned test times. 

It can thus be concluded from the study that the 
statistical procedure can be useful for reliability approx- 
imations of a system, if the components that fail according 
to these non-constant rates do not deviate too significantly 


from a constant failure rate. 
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GRAPHS OF COMPONENT FAILURE RATE FUNCTIONS 
FOR BACH SYSTEM TESTED 
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FIGURE 1 


Component Failure Rate Functions, Z(t), for Case l. 
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FIGURE 2 


Component Failure Rate Functions, z(t), for Case 2. 
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Component Failure Rate Functions, z(t), for Case 3. 
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Component Failure Rate Functions, z(t), for Case 4. 
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FIGURE 5 


Component Failure Rate Functions, 2(t), for Case 5. 
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FIGURE 6 


Component Failure Rate Functions, z(t), for Case 6. 
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FIGURE 7 


Component Failure Rate Functions, Z(t), for Case 7. 
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Component Failure Rate Functions, 
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APPENDIX III 
WEIBULL DISTRIBUTIONS 


The Weibull Probability Distribution is defined by: 


p 
(oes uP? 7 yen (At) O<t<~a 


where i is the "scale" parameter, and 8 is the "shape" 
parameter. 


The Cummulative Distribution Function (CDF) is then 


B 
F(t) = t{ixjdx = = pe) 


and the failure rate function is 


z(t) = f(t)/a(t) = »83t877 


where R(t) = 1 - F(t) = the reliability function. The 
Weibull Probability Distribution reduces to the Exponential 
Probability Distribution when 6 = 1. <Increasime failure 


rates are obtained only when 8 > l. 


onaniel R. Cox, Renewal Theory (London: Methuen and 
Come td 1962 )ep. 2m 
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APPENDIX IV 


LOG NORMAL DISTRIBUTION“ 


The Log Normal Probability Distribution is given by 





wees jin t “4° 
a Ze Oo J 
f(t) = e 0O<t<o 
N2r ot 
= eX 


Where X is distributed as Normal (ger en. For computational 
purposes, it is beneficial to derivesthe Cummulative.Distri- 


bution Function as 


FiG@a= Pi t= ) 
= P( tte ) 
fea ae 
= pj t =a <2 | 
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ppt + | 


then f(t) = P(t) = be ee 


Gt Oo 


The failure rate function is given by: 


1 Afint va 
a ei ae 


z(t) = f(t)/R(t) = 


(Ppaol L. Meyer, Introductory Probability and Sta- 
tistical Applications (Reading, Mass: Addison-Wesley, 1965) 
Dm, LO’. 
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